Design alternatives for scalable Web server accelerators

We study design alternatives for, and describe prototypes and performance of, a scalable and highly available Web server accelerator. The accelerator runs under an embedded operating system and improves Web server performance by caching data. The basic design alternatives include a content router or a TCP router (without content routing) in front of a set of Web cache accelerator nodes, with the cache memory distributed across the accelerator nodes. Content-based routing reduces cache node CPU cycles but can make the front-end router a bottleneck. With the TCP router, a request for a cached object may initially be sent to the wrong cache node; this results in larger cache node CPU cycles, but can provide a higher aggregate through-put, because the TCP router becomes a bottleneck at a higher throughput than the content router. Based on measurement of prototypes, we quantify the throughput ranges in which different designs are preferable. We also examine a combination of content based and TCP routing techniques. We examine optimizations, such as diierent communication and data delivery methods , replication of hot objects, and cache replacement policies that take into account the fact that there might be diierent bottlenecks in the system at diierent times; depending upon which resource is likely to become a bottleneck, a diierent cache replacement algorithm is applied.